COMPLETELY INVARIANT JULIA SETS OF POLYNOMIAL 

SEMIGROUPS 



RICH STANKEWITZ 



Abstract. Let G be a semigroup of rational functions of degree at least two, 
under composition of functions. Suppose that G contains two polynomials with 
non-equal Julia sets. We prove that the smallest closed subset of the Riemann 
sphere which contains at least three points and is completely invariant under 
each element of G, is the sphere itself. 



1. Introduction 

This paper addresses the issue of how one can extend the definition of a Julia 
set of a rational function of a complex variable of degree at least two to the more 
general setting of a Julia set of a rational semigroup. 

It is possible to define the Julia set of a single rational function in two different, 
yet equivalent ways. The first definition of a Julia set is given as the complement 
of the set of normality of the iterates of the given function. The second definition 
of a Julia set is given as the smallest closed completely invariant set which contains 
three or more points. Each of these definitions can be naturally extended to the 
setting of an arbitrary rational semigroup G, but the extensions are not equivalent. 

This paper will show that the extension of the first definition, denoted J(G), 
is better for the purpose of achieving a situation where it is meaningful to study 
dynamics on the components of its complement, the Fatou set. (If one is studying 
dynamics from the point of view that complete invariance is required, then, of 
course, the extension of the second definition, denoted E(G), is better.) We prove 
that for a semigroup generated by two polynomials, of degree greater than or equal 
to two, with non-equal Julia sets, we have E(G) = C, while J(G) is known to be 
a compact subset of the complex plane C. (If the Julia sets of the two generators 
are equal, then both J(G) and E(G) are equal to this common Julia set.) 

2. Definitions and basic facts 

In what follows all notions of convergence will be with respect to the spherical 
metric on the Riemann sphere C. 

A rational semigroup G is a semigroup of rational functions of degree greater 
than or equal to two defined on the Riemann sphere C with the semigroup operation 
being functional composition. When a semigroup G is generated by the functions 
h, ■ ■ ■ » /n> • • • }, we write this as 

G = (fx, hi ■ ■ ■ j /n> • • • )• 
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2 RICH STANKEWITZ 

In p. 360 the definitions of the set of normality, often called the Fatou set, 
and the Julia set of a rational semigroup are as follows: 

Definition 1. For a rational semigroup G we define the set of normality of G, 
N(G), by 

N(G) = {z 6 C : 3 a neighborhood of z on which G is a normal family} 
and define the Julia set of G, J(G), by 

J(G) = C\N(G). 

Clearly from these definitions we see that N(G) is an open set and therefore its 
complement J(G) is a compact set. These definitions generalize the case of iteration 
of a single rational function and we write N((h)) — Nh and J((h)) = J/,. 

Note that J(G) contains the Julia set of each element of G. 

Definition 2. If h is a map of a set Y into itself, a subset X of Y is: 
i) forward invariant under h if h(X) C X; 
m) backward invariant under /i if C X; 

m) completely invariant under h if h(X) C X and C X. 

It is well known that the set of normality of h and the Julia set of h arc completely 
invariant under h (see jl), p. 54), i.e., 

(2.1) h(N h ) = N h = ft-^JV,,) and h(J h ) = J h = h-\j h ). 

In fact we have the following result. 

Property 1. The set Jh is the smallest closed completely invariant (under h) set 
which contains three or more points (see 0, p. 67). 

In fact, this may be chosen as an alternate definition to the definition of Jh given 
in Definition [lj 

From Definition [l], we get that N(G) is forward invariant under each element 
of G and J(G) is backward invariant under each element of G (see p. 360). 
The sets N(G) and J(G) are, however, not necessarily completely invariant under 
the eleme nts of G. This is in contrast to the case of single function dynamics as 
noted in ( [2.l| ). The question then arises, what if we required the Julia set of the 
semigroup G to be completely invariant under each element of G? We consider in 
this paper the consequences of such an extension, given in the following definition. 

Definition 3. For a rational semigroup G we define 

E = E(G) = (~}{S : S is closed, completely invariant under each g G G, #(5") > 3} 

where #(5*) denotes the cardinality of S. 

We note that E(G) exists, is closed, is completely invariant under each element 
of G and contains the Julia set of each element of G by Property |l]. 

We now compare the sets E(G) and J(G). 

Example 1 Suppose that G = (/,<?) and J/ = J g . Then E = Jf = J g since J/ 
is completely invariant under / and J g is completely invariant under g. It is easily 
verified that if Jf — J g , then J(G) — Jf — J g , also. 
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The following example shows that we may have J(G) ^ E(G). 

Example 2 Let a G C, \a\ > 1 and G = (z 2 ,z 2 /a). One can easily show that 
J{G) = {z : 1 < \z\ < \a\} (see §, p.360) while E = C. Note that J z 2 = {z : \z\ = 
1} and J z 2 /a = {z : \z\ = \a\}. 

The main results of this paper will show that, in some sense, Examples 1 and 
2 illustrate the only two possibilities for polynomial semigroups. We prove the 
following theorems. 

Theorem 1. For polynomials f and g of degree greater than or equal to two, Jf ^ 
J g implies E(G) = C where G = (f,g). 

The following theorem follows immediately. 

Theorem 2. For a rational semigroup G' which contains two polynomials f and 
g of degree greater than or equal to two, Jf ^ J g implies E{G') = C. 

3. Auxiliary results 

Suppose that G is a semigroup generated by two polynomials, / and g, of degree 
k and I where k,l > 2. Note that oo ^ J(G), since any small neighborhood of oo 
which is forward invariant under both / and g is necessarily in N(G). This G will 
remain fixed for the rest of the paper. Recall that E(G) D Jf U J g . 

We first begin with a lemma that will give a convenient description of E. Our 
next result shows how E is "built up" from Jf and J g . 

For a collection of sets A, and a function h, we denote new collections of sets by 
h(A) = {h{A) : Ae A} and hr^A) = {h^(A) : A G A}. 

Let us define the following countable collections of sets: 

&0 = {Jfi Jg}, 

£ n +i - r 1 ^) u /(£„) yJg-HSn) U <?(£„), 

oo 

and £ = £ n . 

n=0 

Since E is completely invariant under / and g and contains both Jf and J g , we 
have E D LUef ^- Since E is also closed, we have 



(3.1) 



Ae£ 



where B denotes the closure of the set B. The following lemma shows that these 
two sets are actually equal. 



Lemma 1. We have E = [J Ae£ A. 



Proof. Because of (3.1) we need only show that E C LUef: A. Since the set on the 
right is closed and contains both Jf and J g , it remains only to show that it is also 
completely invariant under both / and g. This can easily be shown by using the 
fact that / and g are continuous open maps. □ 



Corollary 1. The set E has no isolated points; i.e., E is perfect. 
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Proof. Since J/ and J g are perfect (see Q, p. 68) and backward and forward images 
of perfect sets under rational maps are perfect, we see that each set in £ is perfect 
by a routine inductive argument. The corollary then follows since the closure of a 
union of perfect sets is perfect. □ 

Lemma 2. If E has nonempty interior, then E = C. 

Proof. Suppose that E° ^ 0, where B° denotes the interior of the set B. It will be 
shown that this implies the existence of an open set U C E such that U intersects J/ 
or J g . Supposing that U intersects J g , we may observe by the expanding property of 

Julia sets (see Jj|, p. 69) that we have U^°=i 9 n (U) = C By the forward invariance 
of the closed set E under the map g we see that this implies that E = C. 

We will use the following elementary fact: 

For any sets B and C and any function h we have 

(3.2) B n h{C) ± if and only if /i" 1 (B) nC/ 0. 

Since E° ^ 0, there exists an open disc A C E. By Lemma [j]we see then that 
there exists a set A in £ n , say, such that A P\A ^ 0. Since A g £ n , we see that it can 
be expressed as A = h n ■ ■ -hi(J g ), for example, where each hj £ {f,f\g,g }- 
Considering each hj as a map on subsets of C, as opposed to a map on points of 
C, we can define the "inverse" maps h* accordingly, i.e., hi — f implies h\ = f~ x 
and = g^ 1 implies h\ = g. The h* are not true inverses since f~ l (f(X)) may 
properly contain X . 

The fact Q3.2] ) does imply, however, that 

A n A ^ =>- hn ■ ■ ■ hi(J g ) n A ^ 

=^/ ln _ 1 ---^ 1 (j s )n/i;(A)^0 



=>h 1 (J g )nhZ-->h*(A)?Q 
=>J fl nfc; — /£(A)^0. 

Since each h* maps open sets to open sets (as /, g, f~ 1 ,g~ 1 do) we see that 
U = h* ■ ■ ■ h n (A) is open. By the complete invariance of E under both / and g we 
see that each h* takes subsets of E to subsets of E. Hence U C E and the proof of 
Lemma |^ is complete. □ 

Similar to the description of the set E given in Lemma |l| is the following descrip- 
tion of the Julia set J{G) of the semi-group G = (/, g). Consider the countable 
collection of sets 

Ft) = {Jfi Jg}, 

T n+1 = /- 1 (^„)U.g- 1 (J'„), 



and T = T n - 



n=0 



Since J(G) is backward invariant under / and g, closed, and contains both Jf 
and J g (see p. 360), we have J(G) D \J AeJr A. 



Lemma 3. We have J{G) = LUe^^ 
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Proof. Since the set on the right is closed, backward invariant for both / and g (as 
follows as in the proof of Lemma |]) and clearly contains more that three points, it 
must contain J(G) as the complement is then in the set of normality of G. □ 

From Lemma |^, noting that T C £ , and Lemma [l]we get the following result. 
Corollary 2. The Julia set of the semigroup G is contained in E, i.e., J(G) C E. 

Hence, combining Lemma || and Corollary |^, we get the following corollary. 
Corollary 3. // J(G) has nonempty interior, then E = C. 

4. Proof of the main result 

In this section we will prove the main result, Theorem ^, but first we establish 
the necessary lemmas. 

Lemma 4. // / and g are polynomials of degree greater than or equal to two and 
Jf =/= J g , then oo G E. 

Proof. Denoting the unbounded components of the respective Fatou sets of / and g 
by and Goo, we recall (see Q, p. 54 and p. 82) that Jf = d-Fx, and J g = SGoo- 

Since Fxd and Goo are domains with nonempty intersection and <9Foo ^ <9Goo, 
we have J f n Goo ^ or J g n Foo ^ 0- 

Hence we may select z E J g H Fm, say. Denoting the nth iterate of / by we 
see that f n (z) — * oo, and by the forward invariance under the map / of the set E 
we get that each f n (z) £ E. Since E is closed we see then that oo € E. □ 

Remark 1. Since it will be necessary later, we make special note of the fact used 
in the above proof that Jf ^ J g implies Jf n Goo 7^ or J g n F^ 7^ 0. 

Remark 2. Note that the proof above shows also that 00 is not an isolated point 
of E when Jf ^ J g . This, of course, also follows from Corollary |l| and Lemma |J. 

The disc centered at the point z with radius r will be denoted A(z, r). 

Lemma 5. Suppose that A(0, r*) — AUB where A is open, A and B are disjoint, 
and both A and B are nonempty. If both A and B are completely invariant under 
the map L(z) = z J defined on A(0, r*) where < r* < 1 and j > 2, then the set A 
is a union of open annuli centered at the origin and hence B is a union of circles 
centered at the origin. Furthermore, each of A and B contains a sequence of circles 
tending to zero. 

Proof. Let zq = re 10 G A. Since A is open we may choose 8 > such that the arc 
a zo = {re^ :|0-w|<§}cA 

Fix a positive integer n such that j n 5 > 2tt. Since L n (z) = z-? we get 

L«K o )=G(0,W n ) 

where C{z,r) = {£ : £ — z\ = r}. 

By the forward invariance of A under L, we see that G(0, ) C A. But now by 
the backward invariance of A, we get 

G(0,r) =L- n (C(0,r jn )) C A. 



6 



RICH STANKEWITZ 



Thus for any re 10 S A, we have C(0, r) C A. Hence A, being open, must be a 
union of open annuli centered at the origin and therefore B, being the complement 
of A in A(0, r*), must be a union of circles centered at the origin. 

We also note that if C(0, r) C A, then C(0, r J ) C A is a sequence of circles 
tending to zero. Similarly we obtain a sequence of circles in B tending to zero. □ 

Lemma 6. Let L : A(0,r*) — > A(0,r*) ; where < r* < 1, be an analytic function 
such that L(0) = 0. Let B be a set with empty interior which is a union of circles 
centered at the origin and which contains a sequence of circles tending to zero. If 
B is forward invariant under the map L, then L is of the form 

L{z) = az 3 

for some non-zero complex number a and some positive integer j. 
Proof. Since L(0) — 0, we have, near z = 0, 

L(z) = az j + a x z j+1 H 

= az j [l + — z H ] 

for some non-zero complex number a and some positive integer j. 

Let h(z) = L(z)/az 3 and note that h{z) is analytic and tends to 1 as z tends to 
0. We shall prove that h(z) = 1 and the lemma then follows. 

Let C n = C(0, r„) be sequence of circles contained in B with r n — » 0. We 
claim that each L{C n ) is contained in another circle centered at the origin of, say, 
radius r' n . If not, then the connected set L{C n ) would contain points of all moduli 
between, say, r' and r". This, however, would imply that B would contain the 
annulus between the circles C(0, r') and C(0, r"). Thus we have L(C n ) C C(0, r' n ). 

So we see then that h(C n ) C C(0, r' n /\a\r 3 n ). 

But for large n we see that if h were non-constant, then h(C n ) would be a path 
which stays near h(0) = 1 and winds around h(0) — 1. Since h{C n ) is contained 
in a circle centered at the origin, this cannot happen. We thus conclude that h is 
constant. □ 

Lemma 7. If B C A(0, r*) for < r* < 1 is a nonempty relatively closed set which 
is completely invariant under the maps H : z i— > z J and K : z i— » az m defined on 
A(0, r*) where a is a nonzero complex number and j, m are integers with j, m > 2, 
then B = A(0,r*) or \a\ = 1. 

Proof. We may assume that \a\ < 1 by the following reasoning. Suppose that 
\a\ > 1. Let b be a complex number such that b m ~ x — a and define ip(z) = bz. Since 
il)oHoip~ 1 (z) — z J /b 3 and ipoKoip^ 1 ^) — z m , we see that the lemma would then 
imply that ij)(B) = A(0, \b\r*) or |6| = 1. Since we know that ip{B) = A(0, \b\r*) 
exactly when B = A(0,r*), and |6| = 1 exactly when \a\ = 1, we may then assume 
that \a\ < 1. 

We will assume that \a\ < 1 and show that this then implies that B = A(0, r*). 

We first note that by Lemma [|, B is a union of circles centered at the origin and 
B contains a sequence of circles tending to zero. If C(0, p) C B, then by the forward 
invariance of B under H , we see that C(0, p>) C B. Also we get that if C(0, p) C B, 
then by the forward invariance of B under K, we have C(0, |a|p m ) C B. Using a 
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change of coordinates r = log p this invariance can be stated in terms of the new 
functions 

t(r) = jr and s(r) = mr + c 

where c = log \a\ < 0. 

So the action of H and K on A(0, r*) is replaced by the action of t and s on 
I = [— oo,logr*), respectively. In particular, we define 

B' = {logp : C(0, p)cB}U {-00} 

keeping in mind that B is a union of circles centered at the origin. Then 

(4.1) s(B>) c B', 

(4.2) s _1 (B') HI C B', 

(4.3) t(B') C B', 

(4.4) *- x (B')nJ C B', 

(4.5) B' is closed in the relative topology on /. 

In order to make calculations a bit easier we rewrite s(r) = ro + m(r — ro) where 
r = -c/(m - 1) > 0. 
Hence 

s"(r) = r +TO™(r-r ), 
s""(r) =r +TO~"(r-r ), 
t"(r)=j n r, 
*-"(r)=j-"r. 

Consider 

(t-» o s-» o f o s «)(r) = r -ro + ^ + ^- 

j to m j 

Let 

r r r to" + j" - 1 

d n = 1 = ^o ; 

j™ to™ m n j n m n j n 
and note that < d„ < ro with d„ — > as n — > cxd. 

We also note that (t - ™ o o t n o s")(r) = r — ro + d n implies that (s~™ o t - " o 
s n o t n )(r) = r + ro — d n since these two functions are inverses of each other. 

We claim that (— oo,logr* — r ] C B' . 

Let us suppose that this is not the case, and suppose that (r' , f ) is an interval 
disjoint from B' with — 00 < r' < f < logr* — ro. Since B 1 is a closed subset of 
[—00, log r* — ro], we may assume that this interval is expanded so that r' G B'. 
Note that here we used the fact that B contains a sequence of circles going to 0, 
hence B' contains a sequence of points going to —00. 

Let r' n = (t~ n o s~ n o t n o s™)(r') = r' — tq + d n . We claim that each r' n is in B' . 



This is almost obvious from the invariance of B' under s and t in (4.1) through (4.4), 
but some care needs to be taken to insure that each application of s, t, s _1 , and t~ x 
takes points to the right domain. By (fhl|) we see that s(r'), s 2 (r'), . . . , s n (r') e B'. 



Hence by (^3|) we get (t o s")(r'), (t 2 o s")(r'), . . . , (t n o s n )(r') £ B'. 

Since s _1 (r) > r for r e (— 00, ro) we see that because (s~ n ot n os n )(r') is clearly 
less than r' (as t(r) < r for r 6 (—00, 0)), also each of (s _1 o t n o s")(r'), . . . , (s 



o 
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t n o s")(r') must be less than r' < logr*. Hence by (4.2) we see that each of these 
points lies in B' . 

Similarly, since i _1 (r) > r for r £ (—00, 0) and (t~ n os~ n ot n os n )(r') = r' — ?*o + 
d„ < r' < logr* < 0, also each of (r'os^of os n )(r'), • ■ ■ , (r n o S - n ot n os n )(r') 



lies in / = [—00, logr*). Hence by fl4.4| ) each of these points is in B' and so each 
r' n €B'. 

Hence we conclude that r' — ro £ B' since B 1 is relatively closed in / and 
r' n — > r' — r € J. Note also that r^\r' — r . 

Now we claim that for any r" £ -B' (~l (—00, log r* — ro), we have r" + r £ £?'. Let 
= (s~"oi-"os"ot n )(r") = r"+r -d n . Noting that each r^ < r"+r < logr* we 



may again use the invariance of B' under s and t in ( |4.1[ ) through (4.4) in a similar 
fashion as above to obtain that each r" £ B' . Thus also the limit r" + ro £ B' . 

Consider again r' n \r' — r . By applying the above claim to each r' n < r' < 
logr* — r , we get that each r' n + r £ B' . Since r' n + r \r' we then see that we 
have contradicted the statement that (r',f) is disjoint from B'. 

So we conclude that (— 00, logr* — ro] C B'. Clearly then by the partial backward 
invariance of B' under the map t we get [-co, logr*) £ B' . Hence we conclude that 
A(0,r*) = B. □ 

In order to avoid some technical difficulties we will make use of the following 
well known result. 

Theorem A. A polynomial f of degree k is conjugate near 00 to the map z ^ z k 
near the origin. More specifically, there exists a neighborhood U of 00 such that we 
have a univalent 

(j>:U -> A(0,r*) for < r* < 1 with 0(oo) = and <f> o f o 0" 1 (z) = z k . 

Proof. After conjugating /byzn 1/z we may apply Theorem 6.10.1 in p. 150 
to obtain the desired result. □ 

We will denote the conjugate function of / by F, i.e., 

F(z) =4,0 f o4,- 1 (z) = z k . 

In order to further simplify some of the following proofs we will assume that 
4>{U) = D = A(0, r*). Note that U is forward invariant under / since D = A(0, r*) 
is forward invariant under F. We may and will also assume that U is forward 
invariant under g as well. 

We now define a corresponding function for g using the same conjugating map 
as we did for /. Let G be the function defined on D = A(0, r*) given by 

G — (f> o g o (j) . 

Note that G(D) C D. 

Via this change of coordinates, we will use the mappings F and G to obtain 
information about the mappings / and g. In transferring to this simpler coordinate 
system we make the following definitions. 

Let W be the complement of E, i.e., 

W = C\E. 

Note that W is open and it is also completely invariant under both / and g since 
it is the complement of a set which is completely invariant under both / and g. 
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Let W denote the image of W under 0, i.e., W = <j)(U (~l W). Let E' denote the 
image of E under </>, i.e., E' — <j)(U (~l E). Thus W is open and E' is closed in the 
relative topology of D. Note that W' and E' are disjoint since W and E are disjoint 
and 4> is univalent. Also since WUE = C it easily follows that W'UE' = <j)(U) = D. 

By the forward invariance ofWDU under / we see that 

(4.6) f{w') — f o <j)(w r\U) = <f>o f(w nu)c <p{w n u) = W. 

Similarly we get 

(4.7) F(E') C E'. 

Since E' and W are disjoint and forward invariant under F, and since E'UW' — 
D, we see that 

(4.8) F~ 1 {E') HD C E', 

(4.9) F^iW^nD C W. 

Note that in the same way as we obtained the results for F we get 



(4.10) G{W) c W, 

(4.11) G(E') C E', 

(4.12) G-\E') PiD c E', 

(4.13) G _1 (H/')n5cW". 

Lemma 8. If G(z) = az l with \a\ = 1, then Jf = J g . 

Proof. The proof relies on the use of Green's functions. It is well known that the 
unbounded components Frx, and Goo support Green's functions with pole at oo 
which we will denote by G/ and G g respectively. It is also well known that on U 
we have 

G f {z) = -\ag\ct>{z)\ 

since ^ is a map which conjugates / to z i— > z k (see p. 206). 

Since for ip( z ) — bz where b 1 ^ 1 = a, the function tp o cf> conjugates g in U to 
z t—* z , we get in J7, 

G ff ( 2 ) = - log |V> o 0( Z )| = - log = - log |0(«)| 

where the last equality uses the fact that \a\ — 1, and so |6| = 1. 

Hence Gf = G g in U. Since G/ and G g are each harmonic away from oo we get 
that Gf = G g on the unbounded component G of F^ n G^ . 

We claim that this implies that Jf = J g . Assuming that Jf ^ J g , we see by 
Remark [j] that there exists a point which lies in the Julia set of one function, yet in 
the unbounded component of the Fatou set of the other function. Let us therefore 
suppose that z' G J g n F^. Let 7 be a path in Foo connecting z' to 00. We 
see that 7 must intersect dC somewhere, say at z . Since z S Foo n <9G we get 
z G dGoo = Jg. 

We may select a sequence z„£C such that z n — > zq. Since 20 lies on the boundary 
of the domain of the Green's function G 9 , i.e., zo <= >^gj w e have G g {z n ) — » 
(see p. 207). Since zo lies in the domain of the Green's function Gf we see that 
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Gf(z n ) — ► Gf(zo) > 0. We cannot have both of these happen since G f(z n ) = G g (z n ) 
and so we conclude that J g n — 0. Hence we conclude that Jf = J g . □ 

We now are able to prove Theorem [l]. 

Proof of Theorem ^|. Consider whether or not E has nonempty interior. If E° =/= 0, 
then by Lemma || we get E = C. 

If E° = 0, then Lemma || implies that the set W is a union of open annuli 
centered at the origin and hence E' is a union of circles centered at the origin. 
Since E° — 0, the set E' has empty interior. 

Since we know by Remark || that there exists a sequence of points in E tending 
to infinity when Jf ^ J g , also E' must contain a corresponding sequence of circles 
tending to zero. By Lemma ^| we see that the function G is of the form 

G{z) = az l 

for some non-zero complex number a. 

By considering the set E' , we see that Lemma [7] implies that |et| = 1. We see 
that Lemma ^then implies Jf — J g . □ 

References 

1. Alan F. Beardon, Iterations of rational functions, Springer- Verlag, New York, 1991. 

2. A. Hinkkanen and G.J. Martin, The dynamics of semigroups of rational functions I, Proc. 
London Math. Soc. 3 (1996), 358-384. 

Department of Mathematics, University of Illinois, Urbana, Illinois 61801 
Current address: Department of Mathematics, Texas A&M University, College Station, TX 
77843 

E-mail address: richsOmath.tamu.edu 



